The present invention is generally related to a security surveillance system and more particularly to a security surveillance system containing cameras mounted on tall, outdoor poles or other constructions, where wind, thermal and other effects operating on the pole or the construction may cause the exact location and orientation of the camera(s) to change over time. These changes in camera pose are caused by a combination of longer-term, lower-frequency effects (e.g., thermal) and shorter-term, higher-frequency effects (e.g., wind). The term “pole drift” is used herein to denote the longer-term effects, and the term “pole shake” is used to denote shorter term effects.
Image stabilization in video surveillance systems compensates for camera motion for instance caused by wind and vibration effects on the cameras' mounting poles. This compensated camera motion is generally (relatively) high-frequency and translates into apparent pixel motion in the video stream, which in turn will cause false alarms and the detection of large numbers of “spurious” objects. In reality, of course, it is often the camera and not the world that is moving. Tuning the system's motion detection sensitivity parameters so that this motion is filtered out typically also causes much legitimate motion to be filtered out. Most image stabilization algorithms attempt to compensate for this kind of short term pole-induced camera motion.
Camera mounting poles or other mounting structures are also subject to longer-term, lower-frequency effects that will, over the course of minutes or hours, disturb the position and orientation of their attached cameras. The mounting structures in one embodiment of the present invention are substantially fixed or static relative to the ground or a building to which the structure is attached. Under unchanging conditions, a camera that is held by such a structure would generate an image that is stable relative to its environment. Thermal effects such as the heating and cooling of the pole by the sun or absence thereof—are one obvious source of pole motion, which may be relatively slow or longer term effects. Wind and passing heavy vehicles close to a structure for instance, may be a source for fast moving and shorter term effects. However, other effects which may be seasonal, ageing related or based on slow movement of the environment and that affect in a longer term manner a surveillance image are also fully contemplated and can be addressed by the methods and systems provided herein.
These effects operate relatively slowly; generally, they operate beneath the visual threshold of an observer watching video and usually are so slow that they do not significantly affect motion detection. These effects may be cyclical in nature. They may also be permanent.
Aspects of the present invention can address cyclical as well as non-cyclical effects. For instance a non-cyclical effect may be a slow settlement of a new structure to a permanent position. This may be a saturation effect wherein settlement takes place over days or weeks.
Video surveillance systems that convert two-dimensional image data to three-dimensional world data are very affected by these longer-term effects. These longer-term effects actually shift the camera view relative to the camera view at the time the image-to-world coordinate mapping was established and thus can have a large impact on image-to-world coordinate calculations. The system thus loses its ability to accurately place image-plane objects in the real three dimensional world, since the pixels corresponding to the object's location in the image plane no longer represent its correct location in the three-dimensional world.
Accordingly, novel and improved methods and systems that compensate for short term and longer term movement of a camera such as a surveillance camera held on structure, relative to its surveillance environment are required.